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Dialogue-based CALL: an overview 
of existing research 
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Abstract. Dialogue-based Computer-Assisted Language Learning (CALL) covers 
applications and systems allowing a learner to practice the target language in a 
meaning-focused conversational activity with an automated agent. We first present 
a common definition for dialogue-based CALL, based on three features: dialogue as 
the activity unit, computer as the interlocutor and negotiation of outcome through 
open learner production. We then report on a systematic literature review we 
conducted on the main scientific databases which, after filtering, resulted in 138 
relevant papers which were analyzed and coded. Results show a scattered research 
field, with four different disciplinary approaches. We conclude with observations 
regarding the remaining challenges and opportunities for Intelligent CALL (ICALL) 
research. 

Keywords: dialogue-based CALL, conversational agent, dialogue system, chatbot, 
ICALL, literature review. 

1. Introduction 

Second language acquisition theories have long advocated the need for meaning- 
focused activities, especially comprehensible interaction (Long, 1996). It has also 
been amply demonstrated that Synchronous Computer-Mediated Communication 
(SCMC), notably text-based chat, has a positive effect on the development of L2 
proficiency, including speaking proficiency (Lin, 2015). Considering the lack of 
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opportunities for practice with native speakers in most foreign language teaching 
contexts, there has thus been a steady interest in CALL systems that would allow 
learners to practice and develop their communicative skills through interactions in 
natural language with an artificial interlocutor. 

Since the first attempts to tackle this problem, numerous researchers have 
experimented with Natural Language Processing (NLP) techniques to hold 
conversations with learners. However, research on this matter remains scattered 
across different disciplines, with only partial mutual awareness of previous 
works. Previous syntheses have only addressed part of the question, focusing 
either on speech-based applications (Eskenazi, 2009) or on text-based chatbots 
(Fryer & Carpenter, 2006), or mentioning it in ICALL in general (Camper & 
Knapp, 2002). 

The terms used to refer to the systems are not well established either, with important 
variations and multiple possible keywords. Dialogue systems, conversational 
agents and chatbots are sometimes differentiated on modal or technical criteria. 
Kliiwer (2011) distinguishes dialogue systems from chatbots by their “use of 
more theoretically motivated techniques” (p. 3). Jokinen and McTear (2010) treat 
dialogue systems as necessarily spoken, and conversational agents as necessarily 
embodied, raising issues of multimodality and non-verbal communication; 
chatbots are “conversational systems” (i.e. designed for open-ended small talk) 
while dialogue systems are automatically task-oriented. However, the same 
authors recognize a growing convergence of all these systems, which blurs the 
boundaries (Jokinen & McTear, 2010, p. 129). As a result, the three terms are often 
used interchangeably, in an unspecified usage. Bearing in mind that the challenges 
for dialogue management and the opportunities for language learning remain 
essentially equivalent, this is the position we adopted, using dialogue-based CALL 
as an umbrella term. 

More importantly, we first propose an operational definition of dialogue-based 
CALL. This definition served as the main inclusion criteria in the systematic 
literature review. Analyzing the existing research on the topic, we identified the 
most important trends and some challenges for future research. 

1.1. Towards an operational definition 

It is possible to define dialogue-based CALL in a minimal way as (1) dialogue- 
based (2) interactions with a computer for language learning purposes. The 
second element sets us inside tutorial CALL, in contrast with computer-mediated 
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communication, where interactions are with other humans via a computer. 
“Dialogue-based” makes the distinction with item-based approaches, which have 
been dominant in autonomous CALL applications. Nevertheless, this definition 
also applies to any form of interaction that takes the form of a dialogue, such as 
branching dialogues (conversation tree with a limited choice of utterances to select 
from, often used in adventure games) or systems that do not take into account 
what the user has previously uttered (certain question-asking systems). It is thus 
important to state a third characteristic: (3) the interactions must allow a certain 
negotiation of ontcome (Young, 1988) through open learner ontpnt. It is this 
possibility to negotiate the outcome of the interaction, by uttering a free range 
of text or speech, that makes it, at the same time, complex to develop, regarding 
the required natural language processing in the background, and potentially very 
beneficial for language learning, as if enables the learner to freely build his own 
meaning. 

2. Method 

2.1. Collection of a corpns of stndies 

We performed an extensive search on the leading scientific databases (Web of 
Science, Scopus, Proquest), using all the possible keywords referring to dialogue 
systems for language learning, obtaining 604, 494 and 1003 hits respectively (with 
important overlap). We completed this retrieval process through forward citations 
and ancestry search from the previous relevant hits. Only peer-reviewed scientific 
documenfs (journal papers, conference papers and edited book chapters) and 
doctoral dissertations were included in our corpus. We then used the above-stated 
defining features to select papers about dialogue systems only. We discarded papers 
not directly related to language learning (e.g. only mentioning it as a potential 
application). At the end of the inclusion/exclusion process, 109 relevant papers 
remained. 

2.2. Analysis 

11 remaining papers were manually assessed for system, technological and 
evaluation aspects, as well as bibliographical information. These characteristics 
were coded and analyzed mainly in a qualitative manner (see section 4), but 
also in a quantitative way. The frequencies of all the terms referring to dialogue 
systems were computed and used as variables in a Principal Component Analysis 
(PCA). The resulting graph (Figure 2) shows the main variation tendencies, the 
relationships between terms and the projection of the papers on this variation space. 
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3. Results 

As mentioned above, there is a wide diversity of terms used to refer to dialogue- 
based CALL. Out of 109 papers, we identified 49 different terms, ranging from 
agent-mediated language-learning environment to voice-interactive CALL. Some 
papers use more general keywords, which tend to conceal the specificity of the 
system (e.g. virtual world, game), while others coin their own (28 terms are used 
once). The most discriminant terms are intelligent tutoring system, chatbot (or 
chatterbot), conversational agent, and dialogue system. As shown in Figure 1, 
dialogue system is the most frequent (58 papers mentioned it), probably because 
it is associated with an important research strand in natural language processing 
(Jokinen & McTear, 2010). Flowever, what does not appear here is the divergence 
between papers in the use of certain terms. 

Figure 1. Chronological evolution of main key terms used in papers 



The results from the PCA, presented in Figure 2, help us distinguish different 
tendencies and clusters among papers. The fact that chatbot and dialogue system 
are on opposite sides of the y-axis indicates a negative correlation on the second 
principal component: papers mentioning one usually do not (or very rarely) mention 
the other. On the contrary, conversational agent is used globally in a similar way 
as dialogue system. 

Meanwhile, the first principal component (x-axis) seems to correspond to the 
importance of the application for language learning: papers on the positive side 
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of the axis tend to be mostly focused on the technological aspects, while those on 
the left tend to attribute more importance to the analysis of the language learning 
process. In this sense, the latter commonly use terms like tutoring systems and 
ICALL. 


Figure 2. Projection of variables (terms frequencies, as lines) and observations 
(papers) on a bidimensional plot with two principal components (out of 5) 
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These exploratory analyses show there are different clusters of research on dialogue- 
based CALL. Through an interpretative and qualitative analysis, we identified four 
groups of papers, coming from different disciplines. They are presented in Figure 3. 

The first group embraces research on Intelligent Tutoring Systems (ITS). ITS 
existed previously as item-based activities for form-focused practice, but the 
1 980s and 90s saw many efforts to develop more communicative, dialogue-based 
activities as part of ITS. Most papers published in this trend show a particular 
attention to the provision of automated corrective feedback to the learner, but little 
consideration for dialogue management and natural language generation. 

The second group to emerge is related to games and virtual worlds. Many efforts 
are contiguous to ITS, with various systems qualifying as both. The gaming side 
of dialogue systems for language learning has also led to the most important 
commercial applications (e.g. Johnson, 2007). 

The third group encompasses research on (spoken) dialogue systems and 
(embodied) conversational agents. It arises mainly from researchers in NLP who 
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decided to apply these techniques to L2 learning (e.g. Morton, Gunson, & Jack, 
2012). Here, the focus is predominantly on the technological challenges that such 
an endeavor poses. It is probably the research area that presents the most significant 
technological advances and, consequently, it is also the most active. 


The last group, focusing on chatbots, is the most different. Chatbots, the 
descendants of ELIZA (Weizenbaum, 1 966), are text-based conversational systems. 
The diffusion of an accessible framework for programming chatbots, AIML, and 
the popularity of chatbots’ competitions, emulating the Turing test, contributed to 
the creation of countless amateur chatbots. Some research on their application for 
language learning has been conducted (e.g. Coniam, 2014), but research on this 
area regularly fails to connect to the parallel work carried out on dialogue systems. 


Figure 3. Research on dialogue systems for language learning which appeared in 
four different disciplines 
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4. Conclusions 

We have presented an operational definition of dialogue-based CALL, as dialogue- 
based interactions with a computer allowing negotiation of outcome through open 
learner. Such systems have been investigated under various terms and from various 
perspectives in the past, with complementary interests, bridging CALL, natural 


62 


Dialogue-based CALL: an overview of existing research 


language processing, games and artificial intelligence. However, no common 
framework has been made available, and studies of the effectiveness on L2 
learning lack generalizability. Therefore, there is a crucial need for more research 
and development on this topic. 
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